The image processing technology that reconstructs a high-resolution image from multiple low-resolution images, is called the super-resolution processing, and many technologies have been developed conventionally.
For example, as described in Non-Patent Document 1, the typical super-resolution processing methods such as the ML (Maximum-Likelihood) method, the MAP method (Maximum A Posterior) method and the POCS (Projection Onto Convex Sets) method are proposed.
The ML method is a method that defines an evaluation function as square error between the estimated pixel value from an assumed high-resolution image and the actually observed pixel value, and sets a high-resolution image minimizing the evaluation function as an estimated image. In other words, the ML method is a super-resolution processing method based on the principle of maximum likelihood estimation.
Further, the MAP method is a method that estimates the high-resolution image minimizing an evaluation function which added probability information of the high-resolution image to square error. In other words, the MAP method is a super-resolution processing method that estimates the high-resolution image as an optimization problem maximizing posterior probability by using certain prior information for the high-resolution image.
Moreover, the POCS method is a super-resolution processing method that obtains the high-resolution image by generating simultaneous equations about the pixel values of the low-resolution image and the high-resolution image and then solving the simultaneous equations successively.
All of the above-described super-resolution processing methods have the common features of assuming a high-resolution image (an initial high-resolution image), estimating its pixel value for each pixel of all low-resolution images based on a point spread function (PSF) obtained from a camera model from the assumed high-resolution image and then searching for a high-resolution image by minimizing the difference between the estimated value and the observed pixel value (the observed value) Therefore, these super-resolution processing methods are called reconstruction-based super-resolution processing methods.
Since all of the above-described existing reconstruction-based super-resolution processing methods reconstruct a high-resolution image by an iterative computation that needs an initial high-resolution image, the computation cost is very large. Therefore, the reduction of the computation cost was a main problem of existing super-resolution processing methods.
In order to solve this problem, as described in Patent Document 1, inventors of the present invention developed “a fast method of super-resolution processing” that realizes speedup of the super-resolution processing (the iterative reconstruction processing) by reducing the computation cost.
On the other hand, as a high-resolution image generation method reconstructing a high-resolution image from multiple low-resolution images that does not need an iterative computation, for example, there is a method described in Non-Patent Document 2. In the method described in Non-Patent Document 2, although the iterative computation is not performed, the low-resolution image is interpolated to the size of the high-resolution image. A problem that the generated high-resolution image becomes a blurred image by the influence of the interpolation, occurs.